Skip to content

⚡️ Speed up function normalize_host by 11% #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

⚡️ Speed up function normalize_host by 11% #20

wants to merge 1 commit into from

Conversation

@codeflash-ai
Copy link

@codeflash-ai codeflash-ai bot commented Oct 25, 2025

📄 11% (0.11x) speedup for normalize_host in pinecone/utils/normalize_host.py

⏱️ Runtime : 927 microseconds → 834 microseconds (best of 138 runs)

📝 Explanation and details

The optimization combines two separate startswith() calls into a single call that checks both prefixes simultaneously using a tuple: host.startswith(("https://", "http://")).

Key optimization:

  • Reduced method calls: Instead of calling startswith() twice (once for "https://" and once for "http://"), the optimized version calls it only once with a tuple of prefixes
  • Eliminated branching: The original code had two separate if-statements that could both return the same result, while the optimized version handles both cases in a single condition

Why this is faster:

  • str.startswith() with a tuple is optimized at the C level in CPython to check all prefixes in one pass
  • Reduces Python bytecode instructions by eliminating the second conditional check
  • Fewer function calls and branches means less overhead

Performance characteristics:

  • Best speedup (20-21%) occurs with large batches of hosts that have "http://" prefixes, as shown in the large-scale tests
  • Hosts without schemes also benefit significantly (10-15% faster) since they skip the combined check faster
  • Hosts with "https://" prefixes show slight regression in some cases (~9-17% slower) due to the overhead of tuple creation, but this is offset by the overall gains across mixed workloads
  • The 11% overall speedup demonstrates that the optimization works well for typical mixed-input scenarios

The line profiler shows the combined check (line with tuple) executes 3,806 times but only needs one evaluation instead of two, resulting in cleaner execution flow and measurable performance gains.

✅ Correctness verification report:

Test Status
⚙️ Existing Unit Tests 🔘 None Found
🌀 Generated Regression Tests ✅ 5058 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 🔘 None Found
📊 Tests Coverage 100.0%
🌀 Generated Regression Tests and Runtime 
from typing import Optional

# imports
import pytest  # used for our unit tests
from pinecone.utils.normalize_host import normalize_host

# unit tests

# Basic Test Cases
def test_none_input_returns_empty_string():
    # Test that None input returns an empty string
    codeflash_output = normalize_host(None) # 388ns -> 384ns (1.04% faster)

def test_https_prefix_remains_unchanged():
    # Test that a host already starting with https:// is unchanged
    codeflash_output = normalize_host("https://example.com") # 563ns -> 679ns (17.1% slower)

def test_http_prefix_remains_unchanged():
    # Test that a host already starting with http:// is unchanged
    codeflash_output = normalize_host("http://example.com") # 598ns -> 620ns (3.55% slower)

def test_no_scheme_adds_https():
    # Test that a host without a scheme gets https:// prepended
    codeflash_output = normalize_host("example.com") # 672ns -> 646ns (4.02% faster)

def test_no_scheme_with_subdomain():
    # Test that a subdomain host without a scheme gets https:// prepended
    codeflash_output = normalize_host("sub.example.com") # 632ns -> 607ns (4.12% faster)

def test_no_scheme_with_port():
    # Test that a host with a port number and no scheme gets https:// prepended
    codeflash_output = normalize_host("example.com:8080") # 661ns -> 625ns (5.76% faster)

def test_no_scheme_with_path():
    # Test that a host with a path and no scheme gets https:// prepended
    codeflash_output = normalize_host("example.com/path/to/resource") # 675ns -> 620ns (8.87% faster)

def test_no_scheme_with_query():
    # Test that a host with a query string and no scheme gets https:// prepended
    codeflash_output = normalize_host("example.com?query=1") # 653ns -> 624ns (4.65% faster)

def test_no_scheme_with_fragment():
    # Test that a host with a fragment and no scheme gets https:// prepended
    codeflash_output = normalize_host("example.com#fragment") # 618ns -> 605ns (2.15% faster)

# Edge Test Cases

def test_empty_string_returns_https():
    # Test that an empty string returns 'https://'
    codeflash_output = normalize_host("") # 633ns -> 648ns (2.31% slower)

def test_https_in_middle_of_string():
    # Test that a host with 'https://' not at the start gets https:// prepended
    codeflash_output = normalize_host("example.com/https://foo") # 620ns -> 623ns (0.482% slower)

def test_http_in_middle_of_string():
    # Test that a host with 'http://' not at the start gets https:// prepended
    codeflash_output = normalize_host("example.com/http://foo") # 642ns -> 580ns (10.7% faster)

def test_uppercase_scheme_is_not_detected():
    # Test that 'HTTPS://' (uppercase) is not detected and gets https:// prepended
    codeflash_output = normalize_host("HTTPS://example.com") # 650ns -> 610ns (6.56% faster)

def test_mixed_case_scheme_is_not_detected():
    # Test that 'HtTp://' (mixed case) is not detected and gets https:// prepended
    codeflash_output = normalize_host("HtTp://example.com") # 569ns -> 584ns (2.57% slower)

def test_leading_whitespace():
    # Test that leading whitespace causes https:// to be prepended
    codeflash_output = normalize_host("  example.com") # 630ns -> 591ns (6.60% faster)

def test_trailing_whitespace():
    # Test that trailing whitespace causes https:// to be prepended
    codeflash_output = normalize_host("example.com  ") # 588ns -> 588ns (0.000% faster)

def test_host_is_only_whitespace():
    # Test that a string of only whitespace gets https:// prepended
    codeflash_output = normalize_host("   ") # 678ns -> 663ns (2.26% faster)

def test_host_is_special_characters():
    # Test that a host of special characters gets https:// prepended
    codeflash_output = normalize_host("@!$%^&*()") # 615ns -> 611ns (0.655% faster)

def test_host_is_ipv4_address():
    # Test that an IPv4 address gets https:// prepended
    codeflash_output = normalize_host("192.168.1.1") # 627ns -> 588ns (6.63% faster)

def test_host_is_ipv6_address():
    # Test that an IPv6 address gets https:// prepended
    codeflash_output = normalize_host("[2001:db8::1]") # 631ns -> 588ns (7.31% faster)

def test_host_is_localhost():
    # Test that 'localhost' gets https:// prepended
    codeflash_output = normalize_host("localhost") # 617ns -> 581ns (6.20% faster)

def test_host_is_localhost_with_port():
    # Test that 'localhost:5000' gets https:// prepended
    codeflash_output = normalize_host("localhost:5000") # 640ns -> 613ns (4.40% faster)

def test_host_is_dot():
    # Test that '.' gets https:// prepended
    codeflash_output = normalize_host(".") # 687ns -> 668ns (2.84% faster)

def test_host_is_double_dot():
    # Test that '..' gets https:// prepended
    codeflash_output = normalize_host("..") # 658ns -> 584ns (12.7% faster)

def test_host_is_dash():
    # Test that '-' gets https:// prepended
    codeflash_output = normalize_host("-") # 605ns -> 601ns (0.666% faster)

def test_host_is_underscore():
    # Test that '_' gets https:// prepended
    codeflash_output = normalize_host("_") # 639ns -> 597ns (7.04% faster)

def test_host_is_numeric_string():
    # Test that '12345' gets https:// prepended
    codeflash_output = normalize_host("12345") # 652ns -> 572ns (14.0% faster)

def test_host_is_unicode():
    # Test that Unicode characters get https:// prepended
    codeflash_output = normalize_host("例子.测试") # 851ns -> 829ns (2.65% faster)

def test_host_is_long_scheme():
    # Test that a host starting with a long fake scheme gets https:// prepended
    codeflash_output = normalize_host("ftp://example.com") # 654ns -> 657ns (0.457% slower)

# Large Scale Test Cases

def test_large_list_of_hosts_all_no_scheme():
    # Test a large list of hosts without schemes
    hosts = [f"host{i}.example.com" for i in range(1000)]
    for h in hosts:
        codeflash_output = normalize_host(h) # 225μs -> 186μs (20.7% faster)

def test_large_list_of_hosts_with_https():
    # Test a large list of hosts with https://
    hosts = [f"https://host{i}.example.com" for i in range(1000)]
    for h in hosts:
        codeflash_output = normalize_host(h) # 162μs -> 162μs (0.099% faster)

def test_large_list_of_hosts_with_http():
    # Test a large list of hosts with http://
    hosts = [f"http://host{i}.example.com" for i in range(1000)]
    for h in hosts:
        codeflash_output = normalize_host(h) # 203μs -> 167μs (21.5% faster)

def test_large_list_of_none_hosts():
    # Test a large list of None values
    hosts = [None] * 1000
    for h in hosts:
        codeflash_output = normalize_host(h) # 112μs -> 114μs (1.62% slower)

def test_large_list_of_mixed_hosts():
    # Test a large list of mixed hosts
    hosts = []
    expected = []
    for i in range(250):
        hosts.append(None)
        expected.append("")
        hosts.append(f"https://host{i}.com")
        expected.append(f"https://host{i}.com")
        hosts.append(f"http://host{i}.com")
        expected.append(f"http://host{i}.com")
        hosts.append(f"host{i}.com")
        expected.append(f"https://host{i}.com")
    for h, e in zip(hosts, expected):
        codeflash_output = normalize_host(h) # 187μs -> 169μs (11.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from typing import Optional

# imports
import pytest  # used for our unit tests
from pinecone.utils.normalize_host import normalize_host

# unit tests

# -------------------------
# Basic Test Cases
# -------------------------

def test_none_input_returns_empty_string():
    # Test that None input returns empty string
    codeflash_output = normalize_host(None) # 398ns -> 382ns (4.19% faster)

def test_https_prefix_returns_unchanged():
    # Test that https:// prefix is not changed
    codeflash_output = normalize_host("https://example.com") # 531ns -> 584ns (9.08% slower)

def test_http_prefix_returns_unchanged():
    # Test that http:// prefix is not changed
    codeflash_output = normalize_host("http://example.com") # 630ns -> 614ns (2.61% faster)

def test_no_protocol_adds_https():
    # Test that a host without protocol gets https:// added
    codeflash_output = normalize_host("example.com") # 702ns -> 639ns (9.86% faster)

def test_no_protocol_with_subdomain_adds_https():
    # Test that a host with subdomain, no protocol, gets https:// added
    codeflash_output = normalize_host("api.example.com") # 695ns -> 613ns (13.4% faster)

def test_no_protocol_with_port_adds_https():
    # Test that a host with port, no protocol, gets https:// added
    codeflash_output = normalize_host("example.com:8080") # 650ns -> 610ns (6.56% faster)

def test_no_protocol_with_path_adds_https():
    # Test that a host with path, no protocol, gets https:// added
    codeflash_output = normalize_host("example.com/api") # 693ns -> 619ns (12.0% faster)

# -------------------------
# Edge Test Cases
# -------------------------

def test_empty_string_input():
    # Test that empty string input returns https://
    codeflash_output = normalize_host("") # 669ns -> 657ns (1.83% faster)

def test_whitespace_string_input():
    # Test that whitespace string input returns https:// plus whitespace
    codeflash_output = normalize_host("   ") # 686ns -> 605ns (13.4% faster)

def test_https_in_middle_not_prefix():
    # Test that https:// not at prefix gets https:// added
    codeflash_output = normalize_host("example.com/https://foo") # 659ns -> 645ns (2.17% faster)

def test_http_in_middle_not_prefix():
    # Test that http:// not at prefix gets https:// added
    codeflash_output = normalize_host("example.com/http://foo") # 630ns -> 593ns (6.24% faster)

def test_https_uppercase_prefix():
    # Test that HTTPS:// (uppercase) is not considered a prefix, so https:// is added
    codeflash_output = normalize_host("HTTPS://example.com") # 608ns -> 593ns (2.53% faster)

def test_http_uppercase_prefix():
    # Test that HTTP:// (uppercase) is not considered a prefix, so https:// is added
    codeflash_output = normalize_host("HTTP://example.com") # 615ns -> 571ns (7.71% faster)

def test_host_is_only_protocol():
    # Test that input is only 'https://' returns unchanged
    codeflash_output = normalize_host("https://") # 483ns -> 615ns (21.5% slower)
    # Test that input is only 'http://' returns unchanged
    codeflash_output = normalize_host("http://") # 344ns -> 286ns (20.3% faster)

def test_host_is_only_colon_slash_slash():
    # Test that input is only '://' returns https://://
    codeflash_output = normalize_host("://") # 670ns -> 642ns (4.36% faster)

def test_host_is_ip_address():
    # Test that IP address without protocol gets https:// added
    codeflash_output = normalize_host("192.168.1.1") # 690ns -> 606ns (13.9% faster)
    # Test that IP address with http:// returns unchanged
    codeflash_output = normalize_host("http://192.168.1.1") # 404ns -> 353ns (14.4% faster)
    # Test that IP address with https:// returns unchanged
    codeflash_output = normalize_host("https://192.168.1.1") # 187ns -> 204ns (8.33% slower)

def test_host_is_ipv6_address():
    # Test that IPv6 address without protocol gets https:// added
    codeflash_output = normalize_host("[2001:db8::1]") # 668ns -> 596ns (12.1% faster)
    # Test that IPv6 address with http:// returns unchanged
    codeflash_output = normalize_host("http://[2001:db8::1]") # 372ns -> 349ns (6.59% faster)
    # Test that IPv6 address with https:// returns unchanged
    codeflash_output = normalize_host("https://[2001:db8::1]") # 180ns -> 206ns (12.6% slower)

def test_host_with_query_and_fragment():
    # Test that host with query string and fragment gets https:// added
    codeflash_output = normalize_host("example.com?foo=bar#section") # 645ns -> 569ns (13.4% faster)

def test_host_with_unicode_characters():
    # Test that host with unicode characters gets https:// added
    codeflash_output = normalize_host("exámple.cöm") # 895ns -> 778ns (15.0% faster)

def test_host_with_trailing_slash():
    # Test that host with trailing slash gets https:// added
    codeflash_output = normalize_host("example.com/") # 615ns -> 612ns (0.490% faster)

def test_host_with_leading_and_trailing_spaces():
    # Test that host with leading/trailing spaces gets https:// added (spaces preserved)
    codeflash_output = normalize_host("  example.com  ") # 625ns -> 568ns (10.0% faster)

def test_host_with_weird_characters():
    # Test that host with weird characters gets https:// added
    codeflash_output = normalize_host("example.com!@#") # 620ns -> 592ns (4.73% faster)

def test_host_with_multiple_protocols():
    # Test that host with multiple protocol prefixes only checks the first
    codeflash_output = normalize_host("https://http://example.com") # 518ns -> 575ns (9.91% slower)
    codeflash_output = normalize_host("http://https://example.com") # 347ns -> 314ns (10.5% faster)

# -------------------------
# Large Scale Test Cases
# -------------------------







#------------------------------------------------
from pinecone.utils.normalize_host import normalize_host

def test_normalize_host():
    normalize_host('')

def test_normalize_host_2():
    normalize_host(None)

def test_normalize_host_3():
    normalize_host('http://')

def test_normalize_host_4():
    normalize_host('https://')

To edit these changes git checkout codeflash/optimize-normalize_host-mh6guiqq and push.

Codeflash

@codeflash-ai
Optimize normalize_host
5f179c7 
The optimization combines two separate `startswith()` calls into a single call that checks both prefixes simultaneously using a tuple: `host.startswith(("https://", "http://"))`. 

**Key optimization:**
- **Reduced method calls**: Instead of calling `startswith()` twice (once for "https://" and once for "http://"), the optimized version calls it only once with a tuple of prefixes
- **Eliminated branching**: The original code had two separate if-statements that could both return the same result, while the optimized version handles both cases in a single condition

**Why this is faster:**
- `str.startswith()` with a tuple is optimized at the C level in CPython to check all prefixes in one pass
- Reduces Python bytecode instructions by eliminating the second conditional check
- Fewer function calls and branches means less overhead

**Performance characteristics:**
- Best speedup (20-21%) occurs with large batches of hosts that have "http://" prefixes, as shown in the large-scale tests
- Hosts without schemes also benefit significantly (10-15% faster) since they skip the combined check faster
- Hosts with "https://" prefixes show slight regression in some cases (~9-17% slower) due to the overhead of tuple creation, but this is offset by the overall gains across mixed workloads
- The 11% overall speedup demonstrates that the optimization works well for typical mixed-input scenarios

The line profiler shows the combined check (line with tuple) executes 3,806 times but only needs one evaluation instead of two, resulting in cleaner execution flow and measurable performance gains.
@codeflash-ai codeflash-ai bot requested a review from mashraf-222 October 25, 2025 16:00
@codeflash-ai codeflash-ai bot added ⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash labels Oct 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mashraf-222 mashraf-222 Awaiting requested review from mashraf-222

Assignees

No one assigned

Labels

⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant